Image processing apparatus and method, information processing apparatus and method, recording medium, and program

ABSTRACT

The present invention relates to an apparatus and a method for image processing, an apparatus and a method for information processing, a recording medium, and a program that enable appropriate divided display according to a use condition. An image processing apparatus  11  can generate an HD image in which N images are dividedly displayed by setting pixel values of images obtained as a result of image pickup by video cameras  2  in the predetermined HD image using a coordinate conversion table in which coordinates on a display  12 A, coordinates on a small screen for displaying an image obtained as a result of image pickup by a video camera  2 , the small screen including the coordinates on the display  12 A, the coordinates on the small screen corresponding to the coordinates on the display  12 A, and a camera number of the video camera  2  providing the image to be displayed on the small screen are associated with each other, the coordinate conversion table being generated by a coordinate conversion table generating apparatus  13  on the basis of a use condition such as the number of video cameras  2 , arrangement positions of the video cameras  2 , a divided display format, lens distortion of the video camera  2 , or the like.

TECHNICAL FIELD

The present invention relates to an apparatus and a method for imageprocessing, an apparatus and a method for information processing, arecording medium, and a program, and particularly to an apparatus and amethod for image processing, an apparatus and a method for informationprocessing, a recording medium, and a program that enable appropriatedivided display of a plurality of images.

BACKGROUND ART

FIG. 1 shows an example of configuration of a conventional imageprocessing system.

N video cameras 2-1 to 2-N (referred to simply as a video camera 2 whenthe video cameras 2-1 to 2-N do not need to be differentiated from eachother, and the same applies in other cases) disposed at predeterminedpositions with respect to a subject 1 supply an image obtained as aresult of image pickup to an image processing apparatus 3.

The image processing apparatus 3 generates an image in which the Nimages supplied from the video camera 2 are dividedly displayed on adisplay 4A of a display device 4. The image processing apparatus 3supplies the generated image to the display device 4. The display device4 displays the image from the image processing apparatus 3 on thedisplay 4A.

The content of processing of the image processing apparatus 3 differsdepending on the number of video cameras 2 and a divided display format.

For example, the content of processing of the image processing apparatus3 differs between a case where nine (N=9) video cameras 2-1 to 2-9 aredisposed so as to surround a subject 1 as shown in FIG. 2 and nineimages obtained as a result of image pickup by the video camera 2 aredividedly displayed as shown in FIG. 3 and a case where eight (N=8)video cameras 2-1 to 2-8 are disposed such that scenery in alldirections of 360 degrees is a subject 1 as shown in FIG. 4 and eightimages obtained as a result of image pickup by the video camera 2 aredividedly displayed as shown in FIG. 5.

A number attached to a small screen W shown in FIG. 3 and FIG. 5indicates a video camera 2 as a providing source of an image to bedisplayed on the small screen W, and is associated with a numbersucceeding the video camera 2. Specifically, for example, an imageobtained as a result of image pickup by the video camera 2-1 isdisplayed on a small screen W1, and an image obtained as a result ofimage pickup by the video camera 2-2 is displayed on a small screen W2.

In addition, in order to properly display each of the images obtained asa result of image pickup by the video camera 2 on the small screen W,these images need to be corrected on the basis of lens distortion of thevideo camera 2 or the like. However, since the lens distortion or thelike differs between different video cameras 2, the image processingperformed by the image processing apparatus 3 also becomes differentaccordingly.

Thus, the image processing apparatus 3 has been a dedicated apparatusmeeting these conditions. The conventional image processing systemconsequently cannot deal readily with a change in the use condition.

DISCLOSURE OF THE INVENTION

The present invention has been made in view of such a situation, and itis an object of the present invention to make it possible to dealreadily with a change in the use condition.

According to the present invention, there is provided an imageprocessing apparatus including: detecting means for detectingcoordinates on a small screen for displaying a small image includingpredetermined coordinates on a large screen for displaying a large imageincluding a plurality of small images disposed at predeterminedpositions, the coordinates on the small screen corresponding to thepredetermined coordinates on the large screen; reading means for readinga pixel value of a pixel of the predetermined small image, the pixelbeing located at a position corresponding to the coordinates on thesmall screen detected by the detecting means; and outputting means foroutputting the pixel value read by the reading means as a pixel value ofa pixel of the large image, the pixel being located at a positioncorresponding to the predetermined coordinates on the large screen.

The image processing apparatus further includes storing means forstoring a table in which the coordinates on the large screen, thecoordinates on the small screen including the coordinates on the largescreen, the coordinates on the small screen corresponding to thecoordinates on the large screen, and information for identifying thesmall image to be displayed on the small screen are associated with eachother, wherein the detecting means can detect the coordinates on thesmall screen including the predetermined coordinates on the largescreen, the coordinates on the small screen corresponding to thepredetermined coordinates, from the table; and the reading means canread the pixel value of the pixel of the small image identified by theinformation for identifying the small image, the information beingassociated with the predetermined coordinates in the table, the pixelbeing located at the position corresponding to the coordinates on thesmall screen detected by the detecting means.

The small image can be an image corresponding to a picked-up imageobtained as a result of image pickup by an image pickup device.

According to the present invention, there is provided an imageprocessing method including: a detecting step for detecting coordinateson a small screen for displaying a small image including predeterminedcoordinates on a large screen for displaying a large image including aplurality of small images disposed at predetermined positions, thecoordinates on the small screen corresponding to the predeterminedcoordinates on the large screen; a reading step for reading a pixelvalue of a pixel of the predetermined small image, the pixel beinglocated at a position corresponding to the coordinates on the smallscreen detected by a process of the detecting step; and an outputtingstep for outputting the pixel value read by a process of the readingstep as a pixel value of a pixel of the large image, the pixel beinglocated at a position corresponding to the predetermined coordinates onthe large screen.

According to the present invention, there is provided a program on afirst recording medium, the program including: a detecting control stepfor controlling detecting coordinates on a small screen for displaying asmall image including predetermined coordinates on a large screen fordisplaying a large image including a plurality of small images disposedat predetermined positions, the coordinates on the small screencorresponding to the predetermined coordinates on the large screen; areading control step for controlling reading a pixel value of a pixel ofthe predetermined small image, the pixel being located at a positioncorresponding to the coordinates on the small screen detected by aprocess of the detecting control step; and an outputting control stepfor controlling outputting the pixel value read by a process of thereading control step as a pixel value of a pixel of the large image, thepixel being located at a position corresponding to the predeterminedcoordinates on the large screen.

According to the present invention, there is provided a first programthat makes a computer perform a process including: a detecting controlstep for controlling detecting coordinates on a small screen fordisplaying a small image including predetermined coordinates on a largescreen for displaying a large image including a plurality of smallimages disposed at predetermined positions, the coordinates on the smallscreen corresponding to the predetermined coordinates on the largescreen; a reading control step for controlling reading a pixel value ofa pixel of the predetermined small image, the pixel being located at aposition corresponding to the coordinates on the small screen detectedby a process of the detecting control step; and an outputting controlstep for controlling outputting the pixel value read by a process of thereading control step as a pixel value of a pixel of the large image, thepixel being located at a position corresponding to the predeterminedcoordinates on the large screen.

The image processing apparatus and method, and the first programaccording to the present invention detect coordinates on a small screenfor displaying a small image including predetermined coordinates on alarge screen for displaying a large image including a plurality of smallimages disposed at predetermined positions, the coordinates on the smallscreen corresponding to the predetermined coordinates on the largescreen, read a pixel value of a pixel of the predetermined small image,the pixel being located at a position corresponding to the detectedcoordinates on the small screen, and output the read pixel value as apixel value of a pixel of the large image, the pixel being located at aposition corresponding to the predetermined coordinates on the largescreen.

According to the present invention, there is provided an informationprocessing apparatus including: first detecting means for detectingcoordinates on a small screen including coordinates on a large screen,the coordinates on the small screen corresponding to the coordinates onthe large screen; second detecting means for detecting information foridentifying a small image, the information being associated with thecoordinates on the large screen; and generating means for generating afirst table by storing the coordinates on the large screen, thecoordinates on the small screen detected by the first detecting means,and the information for identifying the small image detected by thesecond detecting means in association with each other.

The small image can be an image corresponding to a picked-up imageobtained as a result of image pickup by an image pickup device, and thecoordinates on the large screen can be associated with information foridentifying the image pickup device having a corresponding image pickuparea as the information for identifying the small image in each areacorresponding to the image pickup area of the image pickup device.

The information processing apparatus further includes storing means forstoring a second table in which coordinates on the small screen of thepicked-up image for the small screen which image is cut out from thepicked-up image after being subjected to correction on a basis of acondition of an optical system of the image pickup device andcoordinates on the large screen when the picked-up image for the smallscreen is positioned on the predetermined small screen, the coordinateson the large screen corresponding to the coordinates on the smallscreen, are associated with each other, wherein the first detectingmeans can detect coordinates on the small screen corresponding tocoordinates on the large screen from the second table, and detectcoordinates on the small screen of the picked-up image before thecorrection from the detected coordinates on the small screen.

According to the present invention, there is provided an informationprocessing method including: a first detecting step for detectingcoordinates on a small screen including coordinates on a large screen,the coordinates on the small screen corresponding to the coordinates onthe large screen; a second detecting step for detecting information foridentifying a small image, the information being associated with thecoordinates on the large screen; and a generating step for generating afirst table by storing the coordinates on the large screen, thecoordinates on the small screen detected by a process of the firstdetecting step, and the information for identifying the small imagedetected by a process of the second detecting step in association witheach other.

According to the present invention, there is provided a program on asecond recording medium, the program including: a first detectingcontrol step for controlling detecting coordinates on a small screenincluding coordinates on a large screen, the coordinates on the smallscreen corresponding to the coordinates on the large screen; a seconddetecting control step for controlling detecting information foridentifying a small image, the information being associated with thecoordinates on the large screen; and a generating control step forcontrolling generating a first table by storing the coordinates on thelarge screen, the coordinates on the small screen detected by a processof the first detecting control step, and the information for identifyingthe small image detected by a process of the second detecting controlstep in association with each other.

According to the present invention, there is provided a second programincluding: a first detecting control step for controlling detectingcoordinates on a small screen including coordinates on a large screen,the coordinates on the small screen corresponding to the coordinates onthe large screen; a second detecting control step for controllingdetecting information for identifying a small image, the informationbeing associated with the coordinates on the large screen; and agenerating control step for controlling generating a first table bystoring the coordinates on the large screen, the coordinates on thesmall screen detected by a process of the first detecting control step,and the information for identifying the small image detected by aprocess of the second detecting control step in association with eachother.

The information processing apparatus and method, and the second programaccording to the present invention detect coordinates on a small screenincluding coordinates on a large screen, the coordinates on the smallscreen corresponding to the coordinates on the large screen, detectinformation for identifying a small image, the information beingassociated with the coordinates on the large screen, and generate afirst table by storing the coordinates on the large screen, the detectedcoordinates on the small screen, and the detected information foridentifying the small image in association with each other.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of configuration of aconventional image processing system.

FIG. 2 is a diagram showing an example of arrangement of video camerasin FIG. 1.

FIG. 3 is a diagram showing an example of display of images obtained asa result of image pickup by the video cameras in FIG. 2.

FIG. 4 is a diagram showing another example of arrangement of the videocameras in FIG. 1.

FIG. 5 is a diagram showing another example of display of imagesobtained as a result of image pickup by the video cameras in FIG. 4.

FIG. 6 is a block diagram showing an example of configuration of animage processing system to which the present invention is applied.

FIG. 7 is a diagram showing size of a display of a display device inFIG. 6.

FIG. 8 is a block diagram showing an example of configuration of animage processing apparatus in FIG. 6.

FIG. 9 is a block diagram showing an example of configuration of acoordinate conversion table generating apparatus in FIG. 6.

FIG. 10 is a diagram showing another example of display of imagesobtained as a result of image pickup by the video cameras in FIG. 2.

FIG. 11 is a flowchart of assistance in explaining image processing ofthe image processing apparatus in FIG. 6.

FIG. 12 is a diagram showing an example of a coordinate conversiontable.

FIG. 13 is a flowchart of assistance in explaining operation of thecoordinate conversion table generating apparatus in FIG. 6 whengenerating the coordinate conversion table.

FIG. 14 is a diagram showing an example of a correspondence table ofcoordinates on the display and camera numbers.

FIG. 15 is a diagram showing an example of a correspondence table ofcoordinates on the display and coordinates on small screens of SD imageafter correction.

FIG. 16 is a flowchart of assistance in explaining a process ofgenerating the correspondence table of FIG. 15.

FIG. 17A is a diagram of assistance in explaining a process ofgenerating the correspondence table of FIG. 15.

FIG. 17B is a diagram of assistance in explaining a process ofgenerating the correspondence table of FIG. 15.

FIG. 17C is a diagram of assistance in explaining a process ofgenerating the correspondence table of FIG. 15.

FIG. 18 is a diagram showing an example of arrangement of video camerasin FIG. 6.

FIG. 19 is a diagram showing an example of display of images obtained asa result of image pickup by the video cameras in FIG. 18.

FIG. 20 is a diagram of assistance in explaining image pickup areas ofthe video cameras in FIG. 18.

FIG. 21 is another diagram of assistance in explaining the image pickupareas of the video cameras in FIG. 18.

FIG. 22 is another diagram of assistance in explaining the image pickupareas of the video cameras in FIG. 18.

FIG. 23 is another diagram of assistance in explaining the image pickupareas of the video cameras in FIG. 18.

FIG. 24 is another diagram of assistance in explaining the image pickupareas of the video cameras in FIG. 18.

FIG. 25 is a diagram showing another example of a coordinate conversiontable.

FIG. 26 is a flowchart of assistance in explaining another operation ofthe coordinate conversion table generating apparatus in FIG. 6 whengenerating the coordinate conversion table.

FIG. 27 is a diagram showing another example of a correspondence tableof coordinates on the display and camera numbers.

FIG. is a diagram showing areas on the display corresponding to theimage pickup areas of the video cameras in FIG. 18.

FIG. 29 is another diagram showing areas on the display corresponding tothe image pickup areas of the video cameras in FIG. 18.

FIG. 30 is another diagram showing areas on the display corresponding tothe image pickup areas of the video cameras in FIG. 18.

FIG. 31 is a diagram showing another example of a correspondence tableof coordinates on the display and coordinates on small screens of SDimages after correction.

FIG. 32 is a flowchart of assistance in explaining a process ofgenerating the correspondence table of FIG. 31.

FIG. 33 is a diagram showing an external appearance of anomnidirectional camera.

FIG. 34 is a flowchart of assistance in explaining another operation ofthe coordinate conversion table generating apparatus in FIG. 6 whengenerating the coordinate conversion table.

FIG. 35 is a flowchart of assistance in explaining a process ofgenerating a correspondence table of coordinates on the display andcoordinates on small screens of SD images after correction.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 6 shows an example of configuration of an image processing systemto which the present invention is applied. In the figure, partscorresponding to those in FIG. 1 are identified by the same referencenumerals.

N video cameras 2-1 to 2-N disposed at predetermined positions withrespect to a subject 1 supply an image (for example an SD (standarddefinition) image) obtained as a result of image pickup to an imageprocessing apparatus 11.

The image processing apparatus 11 generates an image (an HD (highdefinition) image) in which the N SD images are dividedly displayed in apredetermined form on a display 12A of a display device 12 from the SDimages supplied from the respective video cameras 2-1 to 2-N using acoordinate conversion table Ta generated by a coordinate conversiontable generating apparatus 13 and provided via a memory card 21. Theimage processing apparatus 11 supplies the generated HD image to thedisplay device 12.

The display device 12 displays the HD image from the image processingapparatus 11 on the display 12A. Incidentally, the display 12A has aresolution of 1920×1080 pixels as shown in FIG. 7.

The coordinate conversion table generating apparatus 13 generates acoordinate conversion table Ta corresponding to use conditions such asthe number of video cameras 2, arrangement positions of the videocameras 2, lens distortion, a display format of the SD images, and thelike. The coordinate conversion table generating apparatus 13 stores thecoordinate conversion table Ta in the memory card 21, for example, toprovide the coordinate conversion table Ta to the image processingapparatus 11.

FIG. 8 shows an example of configuration of the image processingapparatus 11.

N decoders 31-1 to 31-N are supplied with an SD image from thecorresponding video cameras 2-1 to 2-N. The decoders 31-1 to 31-N decodethe SD image input from the video camera 2, and then supply the decodedSD image to corresponding field memories 32-1 to 32-N.

The field memory 32 stores the image data supplied from the decoder 31in field units.

A converting unit 33 reads the image data stored in the field memories32-1 to 32-N as required. The converting unit 33 generates an HD imagein which N images are dividedly displayed in a predetermined form on thedisplay 12A of the display device 12 from the read image data on thebasis of the coordinate conversion table Ta stored in a storage unit 36.The converting unit 33 then supplies the HD image to a frame memory 34.The frame memory 34 stores the HD image supplied from the convertingunit 33.

An encoder 35 reads and encodes image data (image data of the HD image)stored in the frame memory 34 as required. The encoder 35 supplies imagedata obtained as a result to the display device 12.

The storage unit 36 reads the coordinate conversion table Ta from thememory card 21 loaded in the image processing apparatus 11 via aninterface 37, and then stores the coordinate conversion table Ta.

FIG. 9 shows an example of configuration of the coordinate conversiontable generating apparatus 13.

A CPU (Central Processing Unit) 41 is connected with an input/outputinterface 46 via a bus 45. When a command is input from an input unit 48including a keyboard, a mouse and the like via the input/outputinterface 46, the CPU 41 loads into a RAM (Random Access Memory) 43 aprogram (for example, a program for generating the coordinate conversiontable Ta) stored in a ROM (Read Only Memory) 42, on a hard disk 44, oron a recording medium such as a magnetic disk 61, an optical disk 62, amagneto-optical disk 63, a semiconductor memory 64 or the like loadedinto a drive 50, and the CPU 41 executes the program. Further, the CPU41 outputs a result of the process to an output unit 47 including an LCD(Liquid Crystal Display) and the like via the input/output interface 46,for example, as required. Incidentally, the program can be provided fora user integrally with the coordinate conversion table generatingapparatus 13 by being stored on the hard disk 44 or in the ROM 42 inadvance, provided as a packaged medium such as the magnetic disk 61, theoptical disk 62, the magneto-optical disk 63, the semiconductor memory64 or the like, or provided from a satellite, a network or the like tothe hard disk 44 via a communication unit 49.

The CPU 41 stores the generated coordinate conversion table Ta in thememory card 21 via the drive 50.

Description will next be made of operation of the image processingapparatus 11 in a case where nine (N=9) video cameras 2-1 to 2-9 aredisposed so as to surround a subject 1 at a relatively close range asshown in FIG. 2 and images obtained as a result of image pickup by thevideo camera 2 are dividedly displayed as shown in FIG. 10 (such a usecondition will hereinafter be referred to as a first use condition asappropriate) with reference to a flowchart of FIG. 11.

In step S1, the converting unit 33 of the image processing apparatus 11reads a coordinate conversion table Ta as shown in FIG. 12 which tableis stored in the storage unit 36.

In the coordinate conversion table Ta, coordinates on the display 12A,coordinates on a small screen W including the coordinates on the display12A, the coordinates on the small screen W corresponding to thecoordinates on the display 12A, and a camera number assigned to thevideo camera 2 providing an image to be displayed on the small screen Ware associated with each other. In addition, information indicating thatblack color is to be outputted (information represented by “black coloroutput” in FIG. 12) is set to predetermined coordinates on the display12A. Incidentally, coordinates on the display 12A associated with theinformation indicating that black color is to be outputted are notassociated with coordinates on the small screen W or a camera number.

Next, in step S2, the converting unit 33 selects coordinates of onepixel on the display 12A (coordinates of one pixel forming an HD image).

In step S3, the converting unit 33 determines whether or not theinformation indicating that black color is to be outputted is associatedwith the coordinates on the display 12A selected in step S2 in thecoordinate conversion table Ta. When the converting unit 33 determinesthat the information indicating that black color is to be outputted isnot associated with the coordinates on the display 12A, the processproceeds to step S4 to detect a camera number associated with theselected coordinates on the display 12A from the coordinate conversiontable Ta.

Next, in step S5, the converting unit 33 selects the field memory 32corresponding to the video camera 2 assigned the camera number detectedin step S4.

In step S6, the converting unit 33 detects coordinates on the smallscreen W associated with the coordinates on the display 12A selected instep S2 from the coordinate conversion table Ta.

Next, in step S7, the converting unit 33 reads a pixel value of a pixelof an SD image stored in the field memory 32 selected in step S5 at aposition corresponding to the coordinates on the small screen W detectedin step S6, and stores the pixel value as a pixel value to be output atthe coordinates on the display 12A selected in step S2 in the framememory 34.

When the converting unit 33 determines in step S3 that the informationindicating that black color is to be outputted is set, the processproceeds to step S8, where the converting unit 33 stores a pixel valuedisplayed as black color as a pixel value to be output at thecoordinates on the display 12A selected in step S2 in the frame memory34.

For example, when predetermined coordinates within a small screen W5 inFIG. 10 are selected in step S2, the selected coordinates are associatedwith the camera number of the video camera 2-5 and are not associatedwith the information indicating that black color is to be outputted inthe coordinate conversion table Ta, and therefore the camera number ofthe video camera 2-5 is detected (step S3 and step S4). Next, the fieldmemory 32-5 corresponding to the video camera 2-5 is selected (step S5),and the coordinates on the small screen W associated with the selectedcoordinates on the display 12A are detected from the coordinateconversion table Ta (step S6).

Then the pixel value of the pixel of the SD image (step S5) from thevideo camera 2-5 at the position corresponding to the detectedcoordinates on the small screen W (step S6) is stored as the pixel valueto be output at the selected coordinates on the display 12A (step S7).

For example, when coordinates of a screen other than small screens W1 toW9 in FIG. 10 (a hatched part in the figure) are selected in step S2,since the information indicating that black color is to be outputted isset to such coordinates in the coordinate conversion table Ta (step S3),the pixel value displayed as black color is stored as the pixel value tobe output at the selected coordinates on the display 12A (step S8).

Returning to FIG. 11, when the predetermined pixel value is stored inthe frame memory 34 in step S7 or step SB, the process of step S9, wherethe converting unit 33 determines whether or not all coordinates on thedisplay 12A are selected. When the converting unit 33 determines thatthere remain coordinates yet to be selected, the process returns to stepS2 to select next coordinates on the display 12A.

When the converting unit 33 determines in step S9 that all thecoordinates are selected, the process is ended.

The coordinate conversion table Ta shown in FIG. 12 is generated by thecoordinate conversion table generating apparatus 13 on the basis of thefirst use condition and lens distortion of the video camera 2 as laterdescribed. Therefore the image processing apparatus 11 in this systemperforms the process as described above according to the coordinateconversion table Ta, whereby the nine SD images supplied from the videocamera 2 can be dividedly displayed as shown in FIG. 10.

Description will next be made of operation of the coordinate conversiontable generating apparatus 13 when generating the coordinate conversiontable Ta used in the above process with reference to a flowchart of FIG.13.

In step S21, the CPU 41 of the coordinate conversion table generatingapparatus 13 selects coordinates of one pixel on the display 12A of thedisplay device 12 (coordinates of one pixel forming an HD image) (FIG.10).

In step S22, the CPU 41 determines whether or not the coordinates on thedisplay 12A selected in step S21 are within a small screen W (within ascreen displaying an image corresponding to an SD image obtained as aresult of image pickup by the video camera 2).

Specifically, in this case, the hard disk 44 stores a correspondencetable Tb1 as shown in FIG. 14 in which coordinates within the smallscreens W1 to W9 are associated with the camera number of the videocamera 2 providing an image to be displayed at the coordinates, andcoordinates within the screen other than the small screens W1 to W9 (thehatched part in FIG. 10) are associated with information indicating thescreen other than the small screens W1 to W9 (hereinafter referred to asoutside display area information). Referring to the correspondence tableTb1, the CPU 41 determines whether or not the coordinates on the display12A selected in step S21 are associated with the camera number of thevideo camera 2.

When the CPU 41 determines in step S22 that the coordinates on thedisplay 12A selected in step S21 are within a small screen W (when thecoordinates selected in step S21 are associated with the camera numberof the video camera 2 in the correspondence table Tb1), the processproceeds to step S23, where the CPU 41 detects the camera number of thevideo camera 2 corresponding to the coordinates on the display 12Aselected in step S21 from the correspondence table Tb1.

Next, in step S24, the CPU 41 detects coordinates on the small screen Wassociated with the coordinates on the display 12A selected in step S21from a correspondence table Tb2 stored in the hard disk 44 which tableindicates correspondences between coordinates on the display 12A shownin FIG. 15 and coordinates on the small screen W of an image for thesmall screen W which image is cut out from the SD image obtained as aresult of image pickup by the video camera 2 and corrected on the basisof lens distortion of the video camera 2 or the like as later described.

In step S25, the CPU 41 calculates coordinates on the small screen W ofthe SD image before correction (the original SD image) as laterdescribed from the coordinates on the small screen W detected in stepS24.

Next, in step S26, the CPU 41 stores, in the coordinate conversion tableTa (FIG. 12), the coordinates on the display 12A selected in step S21,the camera number detected in step S23, and the coordinates on the smallscreen W calculated in step S25 in association with each other.

When the CPU 41 determines in step S22 that the coordinates on thedisplay 12A selected in step S21 are not within a small screen W (whenthe coordinates on the display 12A selected in step S21 are associatedwith the outside display area information in the correspondence tableTb1), the process proceeds to step S27.

In step S27, the CPU 41 stores, in the coordinate conversion table Ta(FIG. 12), the coordinates on the display 12A selected in step S21 andthe information indicating that black color is to be outputted inassociation with each other.

When the predetermined information is stored in the coordinateconversion table Ta in step S26 or step S27, the process proceeds tostep S28, where the CPU 41 determines whether or not all the coordinateson the display 12A are selected on the display 12A. When the CPU 41determines that there remain coordinates yet to be selected, the processreturns to step S21 to select next coordinates and perform thesubsequent process.

When the CPU 41 determines in step S28 that all the coordinates areselected, the process is ended.

Description will next be made of a process of generating thecorrespondence table Tb2 (FIG. 15) used when the coordinate conversiontable Ta (FIG. 12) is generated with reference to a flowchart of FIG.16. Incidentally, suppose in this case that the coordinate conversiontable generating apparatus 13 generates the correspondence table Tb2.

In step S41, the CPU 41 of the coordinate conversion table generatingapparatus 13 selects one camera number from the camera numbers of thevideo cameras 2-1 to 2-9.

Next, in step S42, assuming an SD image having the same size as the SDimage obtained as a result of image pickup by the video camera 2 (animage having the same size as a small screen W in this case), the CPU 41corrects coordinates (Xa, Ya) on the small screen W of each pixelforming the SD image (coordinates on the small screen W of the SD imagebefore correction) according to Equation (1) to thereby calculatecoordinates (Xb, Yb) (coordinates on the small screen of the SD imageafter correction). Incidentally, by thus converting the coordinates, ablurred peripheral portion of the SD image resulting from lensdistortion of the video camera 2 can be made clear.Xb=Xac+(Xa−Xac)(1+k ₁ ×r ² +k ₂ ×r ⁴)Yb=Yac+(Ya−Yac)(1+k ₁ ×r ² +k ₂ ×r ⁴)r=((Xa−Xac)+(Ya−Yac)²)^(1/2)  (1)

In the equation, the coordinates (Xac, Yac) are coordinates of a pixellocated at a center of the SD image. k1 and k2 are coefficientsdetermined on the basis of lens distortion of the video camera 2 or thelike. r is a distance between the pixel at the coordinates (Xa, Ya) anda pixel at a center of distortion of the SD image.

In step S43, the CPU 41 determines an area of the SD image obtained as aresult of the correction in step S42 which area corresponds to the sizeof the small screen W. In step S44, the CPU 41 selects coordinates (Xb,Yb) on the small screen W of each pixel of the SD image within thedetermined area. That is, coordinates on the small screen W of the SDimage after the correction to be displayed on the small screen W areselected.

The SD image obtained as a result of image pickup by the video camera 2(FIG. 17A) is enlarged and deformed into a shape of a bobbin as shown bya solid line in FIG. 17B, for example, by the correction according toEquation (1). From the SD image, the coordinates within the areadetermined in step S43 as shown by a dotted line in FIG. 17B areselected (FIG. 17C). Incidentally, coordinates (Xai, Yai) in FIG. 17Arepresent arbitrary coordinates (Xa, Ya), and coordinates (Xbi, Ybi) inFIG. 17B and FIG. 17C represent coordinates (Xb, Yb) obtained as aresult of correction of the coordinates (Xai, Yai).

In step S45, the CPU 41 selects coordinates of one pixel from thecoordinates on the small screen W of the SD image after the correction,the coordinates on the small screen W of the SD image after thecorrection being selected in step S44. In step S46, the CPU 41 convertsthe coordinates into coordinates on the display 12A.

Specifically, the coordinates on the small screen W selected in step S45are converted into corresponding coordinates on the display 12A when theSD image of the area determined in step S43 (FIG. 17C) is mapped ontothe small screen W for displaying the image of the video camera 2 havingthe camera number selected in step S41.

For example, in a case where the camera number of the video camera 2-5is selected in step S41, the coordinates on the small screen W selectedin step S45 are converted into corresponding coordinates on the display12A when the SD image of the area determined in step S43 is mapped ontothe small screen W5.

Next, in step S47, the CPU 41 stores, in the correspondence table Tb2(FIG. 15), the coordinates on the small screen W of the SD image afterthe correction, the coordinates on the small screen W of the SD imageafter the correction being selected in step S45, and the coordinates onthe display 12A obtained in step S46 in association with each other.

In step S48, the CPU 41 determines whether or not all the coordinatesselected in step S44 are selected. When the CPU 41 determines that thereremain coordinates yet to be selected, the process returns to step S45to select next coordinates and perform the subsequent process.

When the CPU 41 determines in step S48 that all the coordinates areselected, the process proceeds to step S49, where the CPU 41 determineswhether or not all the camera numbers are selected. When the CPU 41determines that there remains a camera number yet to be selected, theprocess returns to step S41 to select the next camera number.

When the CPU 41 determines in step S49 that all the camera numbers areselected, the process is ended.

Thus, a pixel value output at given coordinates Ph (not shown) on thedisplay 12A within the small screen W5, for example, is a pixel value ofa pixel located at coordinates (Xai, Yai) on the small screen W (FIG.17A) (step S25 in FIG. 13) of the SD image obtained as a result of imagepickup by the video camera 2-5 which coordinates (Xai, Yai) becomecoordinates (Xbi, Ybi) on the small screen W (FIG. 17B) (step S24 inFIG. 13) detected as coordinates corresponding to the coordinates Ph onthe display 12A as a result of correction in step S42 in FIG. 16.

That is, without the need for calculating coordinates of the input SDimage obtained as a result of image pickup by the video camera 2 whichcoordinates accommodate lens distortion of the video camera 2 orcalculating coordinates on the display 12A according to a divideddisplay format, the image processing apparatus 11 can generate an HDimage dividedly displaying the SD image obtained as a result of imagepickup by the video camera 2 and subjected to distortion correction byonly setting pixel values of the SD image in the HD image according tothe coordinate conversion table Ta as shown in FIG. 11.

Since the coordinates (Xb, Yb) on the display 12A of the SD image afterthe correction in the correspondence table Tb2 (FIG. 15) are calculatedaccording to Equation (1) as described above, reverse correction isperformed by optimization corresponding to the equation to calculate thecoordinates on the small screen W of the SD image before the correctionin step S25 in FIG. 13.

Description will next be made of operation of the image processingapparatus 11 in a case where nine video cameras 2-1 to 2-9 are disposedso as to form a plane in a vertical direction with respect to a distantsubject 1 as shown in FIG. 18 and images obtained as a result of imagepickup by the video camera 2 are dividedly displayed as shown in FIG. 19(such a use condition will hereinafter be referred to as a second usecondition as appropriate).

In this example, the video cameras 2 are disposed such that image pickupareas of the video cameras 2 adjacent to each other overlap each otheras indicated by frames of dotted lines in FIG. 20. For example, theimage pickup area of the video camera 2-1 (a frame of a solid line inFIG. 21) overlaps the image pickup area of the video camera 2-2 (a frameof a solid line in FIG. 22), the image pickup area of the video camera2-4 (a frame of a solid line in FIG. 23), and the image pickup area ofthe video camera 2-5 (a frame of a solid line in FIG. 24).

The image processing apparatus 11 performs the process shown in theflowchart of FIG. 11 as in the case of the first use form, using acoordinate conversion table Ta as shown in FIG. 25 generated by thecoordinate conversion table generating apparatus 13 on the basis of thesecond use condition and the lens distortion of the video camera 2 orthe like. That is, although values of coordinates and the like handledin the process of the image processing apparatus 11 are changed, theprocess of the image processing apparatus 11 in the case of the firstuse condition and the process of the image processing apparatus 11 inthe case of the second use condition are essentially the same. Thereforeit is possible to deal readily with a change in the use condition.

Incidentally, unlike the case of the first use condition (FIG. 10), anHD image is displayed on the whole of the display 12A of the displaydevice 12 in a display format of FIG. 19. Therefore, unlike the case ofthe first use condition (FIG. 12), information indicating that blackcolor is to be outputted is not set in the coordinate conversion tableTa. Thus, in this example, a determination of YES is not made in step S3in the flowchart of FIG. 11, and therefore the storing of a pixel valuefor outputting black color is not performed in the process of FIG. 8.

Description will next be made of operation of the coordinate conversiontable generating apparatus 13 when generating the coordinate conversiontable Ta (FIG. 25) used in this example with reference to a flowchart ofFIG. 26.

In step S61, the CPU 41 of the coordinate conversion table generatingapparatus 13 selects coordinates of one pixel on the display 12A of thedisplay device 12 (coordinates of one pixel forming an HD image) (FIG.19). In step S62, the CPU 41 selects one camera number.

In step S63, the CPU 41 determines whether or not the coordinates on thedisplay 12A selected in step S61 are associated with the camera numberselected in step S62 in a correspondence table Tc1 shown in FIG. 27which table shows correspondences between coordinates on the display 12Aand camera numbers.

Coordinates on the display 12A in the correspondence table Tc1 shown inFIG. 27 are associated with the camera numbers of the video cameras 2having the image pickup areas as shown in FIG. 20 corresponding to areasas shown in FIG. 28.

For example, coordinates within the area Q1 (FIG. 29) corresponding tothe image pickup area of the video camera 2-1 (FIG. 21) on the display12A are associated with the camera number of the video camera 2-1.Coordinates within the area Q2 (FIG. 30) corresponding to the imagepickup area of the video camera 2-2 (FIG. 22) on the display 12A areassociated with the camera number of the video camera 2-2. Incidentally,coordinates on the display 12A belonging to both the area Q1 and thearea Q2 (for example coordinates (Xhi, Yhi) in FIG. 29 and FIG. 30) areassociated with the two camera numbers of the video cameras 2-1 and 2-2.

Returning to FIG. 26, when the CPU 41 determines in step S63 that thecoordinates on the display 12A selected in step S61 are associated withthe camera number selected in step S62, the process proceeds to stepS64.

In step S64, the CPU 41 detects coordinates on a small screen Wassociated with the coordinates on the display 12A selected in step S61and the camera number selected in step S62 from a correspondence tableTc2 stored on the hard disk 44 and shown in FIG. 31 which table showscorrespondences between coordinates on the display 12A, coordinates onthe small screen W of an image for the small screen W which image is cutout from an SD image obtained as a result of image pickup by the videocamera 2 and corrected on the basis of lens distortion of the videocamera 2 or the like, and the camera number of the video camera 2 havingan image pickup area corresponding to coordinates on the display 12A.Incidentally, a method of generating the correspondence table Tc2 willbe described later.

Next, in step S65, the CPU 41 calculates coordinates on the small screenW of the SD image before the correction from the coordinates on thesmall screen W of the SD image after the correction detected in stepS64.

When the CPU 41 determines in step S63 that the coordinates on thedisplay 12A selected in step S61 are not associated with the cameranumber selected in step S62, or when the coordinates on the small screenW of the SD image before the correction are calculated in step S65, theprocess proceeds to step S66, where the CPU 41 determines whether or notall the camera numbers are selected. When the CPU 41 determines thatthere remains a camera number yet to be selected, the process returns tostep S62 to select the next camera number.

When the CPU 41 determines in step S66 that all the camera numbers areselected, the process proceeds to step S67, where the CPU 41 determineswhether or not the coordinates on the display 12A selected in step S61are associated with a plurality of camera numbers in the correspondencetable Tc1 (FIG. 27). When the CPU 41 determines that the coordinates onthe display 12A selected in step S61 are associated with a plurality ofcamera numbers, the process proceeds to step S68.

For example, the coordinates (Xhi, Yhi) on the display 12A shown in FIG.29 and FIG. 30 are associated with both the camera number of the videocamera 2-1 and the camera number of the video camera 2-3 in thecorrespondence table Tc1. Therefore, when the coordinates (Xhi, Yhi) areselected in step S61, the process proceeds to step S68.

In step S68, the CPU 41 determines coordinates on one small screen W tobe set in the coordinate conversion table Ta from coordinates on smallscreens W of a plurality of SD images before correction (step S65) whichcoordinates are calculated by performing the process of steps S62 to S65a plurality of times (for example coordinates calculated when the cameranumber of the video camera 2-1 is selected in step S62 and coordinatescalculated when the camera number of the video camera 2-2 is selected instep S62).

For example, when the coordinates (Xhi, Yhi) (FIGS. 29 and 30) arecloser to a center of the area Q1 than a center of the area Q2,coordinates on the display 12A of an SD image before correction whichcoordinates are calculated when the camera number of the video camera2-1 is selected are selected.

When the CPU 41 determines in step S67 that the coordinates on thedisplay 12A selected in step S61 are not associated with a plurality ofcamera numbers in the correspondence table Tc1, or when coordinates onone small screen W are determined in step S68, the process proceeds tostep S69.

In step S69, the CPU 41 stores the coordinates on the display 12Aselected in step S61, the coordinates on the small screen W calculatedin step S65 when the coordinates on the display 12A selected in step S61are not associated with a plurality of camera numbers or the coordinateson the small screen W determined in step S68 when the coordinates on thedisplay 12A selected in step S61 are associated with a plurality ofcamera numbers, and the camera number selected in step S62 when thecoordinates on the small screen W are calculated, in association witheach other in the coordinate conversion table Ta (FIG. 25).

Next, in step S70, the CPU 41 determines whether or not all coordinateson the display 12A are selected. When the CPU 41 determines that thereremain coordinates yet to be selected, the process returns to step S61to select next coordinates and perform the subsequent process.

When the CPU 41 determines in step S70 that all the coordinates areselected, the process is ended.

Description will next be made of a process of generating thecorrespondence table Tb2 (FIG. 31) used when the coordinate conversiontable Ta (FIG. 25) corresponding to the second use condition isgenerated with reference to a flowchart of FIG. 32. Incidentally,suppose in this case that the coordinate conversion table generatingapparatus 13 generates the correspondence table Tb2.

In step S81, the CPU 41 of the coordinate conversion table generatingapparatus 13 selects one camera number.

Next, in step S82, assuming an SD image having the same size as the SDimage obtained as a result of image pickup by the video camera 2, theCPU 41 corrects coordinates (Xa, Ya) on the small screen W of each pixelforming the SD image (coordinates on the small screen W of the SD imagebefore correction) according to Equation (1) to thereby calculatecoordinates (Xb, Yb) (coordinates on the small screen of the SD imageafter correction).

In step S83, the CPU 41 determines an area of the SD image obtained as aresult of the correction in step S82 which area corresponds to the sizeof the small screen W. In step S84, the CPU 41 selects coordinates (Xb,Yb) on the small screen W of each pixel of the SD image within thedetermined area. That is, coordinates on the small screen W of the SDimage after the correction to be displayed on the small screen W areselected.

Next, in step S85, the CPU 41 selects coordinates of one pixel from thecoordinates on the small screen W of the SD image after the correction,the coordinates on the small screen W of the SD image after thecorrection being selected in step S84. In step S86, the CPU 41 convertsthe coordinates into coordinates on the display 12A.

Specifically, the coordinates on the small screen W selected in step S85are converted into corresponding coordinates on the display 12A when theSD image of the area determined in step S83 is mapped onto the smallscreen W for displaying the image of the video camera 2 having thecamera number selected in step S81.

For example, in a case where the camera number of the video camera 2-5is selected in step S81, the coordinates on the small screen W selectedin step S85 are converted into corresponding coordinates on the display12A when the SD image of the area determined in step S83 is mapped ontothe small screen W5 (FIG. 19).

Next, in step S87, the CPU 41 stores the camera number selected in stepS81, the coordinates on the small screen W of the SD image after thecorrection, the coordinates on the small screen W of the SD image afterthe correction being selected in step S85, and the coordinates on thedisplay 12A obtained in step S86 in association with each other in thecorrespondence table Tc2 as shown in FIG. 31.

In step S88, the CPU 41 determines whether or not all the coordinatesselected in step S84 are selected. When the CPU 41 determines that thereremain coordinates yet to be selected, the process returns to step S85to select next coordinates.

When the CPU 41 determines in step S88 that all the coordinates areselected, the process proceeds to step S89, where the CPU 41 determineswhether or not all the camera numbers are selected. When the CPU 41determines that there remains a camera number yet to be selected, theprocess returns to step S81 to select the next camera number.

When the CPU 41 determines in step S89 that all the camera numbers areselected, the process is ended.

Description will next be made of operation of the image processingapparatus 11 in a case where eight (N 8) video cameras 2-1 to 2-8 aredisposed such that scenery in all directions of 360 degrees is a subject1 as shown in FIG. 4 and images obtained as a result of image pickup bythe video camera 2 are dividedly displayed as shown in FIG. 5 (such ause condition will hereinafter be referred to as a third use conditionas appropriate).

Incidentally, the eight video cameras 2 shown in FIG. 4 are in practicearranged at a predetermined angle in an omnidirectional camera 101 asshown in FIG. 33 such that centers of projection of the video cameras 2coincide near a center of the omnidirectional camera 101 and directionsof lines of sight of the video cameras 2 are in one horizontal plane.

Plane mirrors 110 are disposed in the directions of the lines of sightof the video cameras 2.

That is, the video cameras 2 pick up an image of surrounding sceneryreflected by the corresponding plane mirrors 110, whereby theomnidirectional camera 101 as a whole can pick up an image of thescenery in 360 degrees in a horizontal direction.

The image processing apparatus 11 performs the process shown in theflowchart of FIG. 11 as in the case of the first use condition and thesecond use condition, using a coordinate conversion table Ta generatedby the coordinate conversion table generating apparatus 13 on the basisof the third use condition and the lens distortion of the video camera 2or the like. As a result, eight images obtained as a result of imagepickup by the video camera 2 can be dividedly displayed as shown in FIG.5.

Description will next be made of operation of the coordinate conversiontable generating apparatus 13 in generating the coordinate conversiontable Ta used in this example with reference to a flowchart of FIG. 34.

A process performed in steps S101 to S104 is the same as in steps S61 toS64 in FIG. 26, and therefore description thereof will be omitted.

In step S105, the CPU 41 optimizes coordinates on a small screen W of anSD image after correction detected in step S104 and performs a processreverse to a mirror inversion process (a reverse mirror inversionprocess) to thereby calculate coordinates on the small screen W of theSD image before the correction.

A process performed in steps S106 to S110 is the same as in steps S66 toS70 in FIG. 26, and therefore description thereof will be omitted.

Description will next be made of a process of generating acorrespondence table used in step S104 which table shows correspondencesbetween coordinates on the display 12A and coordinates on the smallscreen W of the SD image after the correction with reference to aflowchart of FIG. 35. Incidentally, suppose in this case that thecoordinate conversion table generating apparatus 13 generates thecorrespondence table.

In step S121, the CPU 41 of the coordinate conversion table generatingapparatus 13 selects one camera number.

Next, in step S122, assuming an SD image having the same size as the SDimage obtained as a result of image pickup by the video camera 2, theCPU 41 corrects coordinates (Xa, Ya) of each pixel forming the SD imageaccording to Equation (1) to thereby calculate coordinates (Xb, Yb).Also, the CPU 41 performs the mirror inversion process.

A process performed in steps S123 to S129 is the same as in steps S83 toS89 in FIG. 32, and therefore description thereof will be omitted.

The series of processes described above can be implemented not only byhardware but also by software. When the series of processes is to beimplemented by software, a program forming the software is installedonto a computer, and the computer executes the program, whereby theimage processing apparatus 11 and the coordinate conversion tablegenerating apparatus 13 described above are functionally implemented.

It is to be noted that in the present specification, the stepsdescribing the program provided by a recording medium include not onlyprocesses carried out in time series in the described order but alsoprocesses carried out in parallel or individually and not necessarily intime series.

Also, in the present specification, a system refers to an apparatus as awhole formed by a plurality of apparatus.

INDUSTRIAL APPLICABILITY

According to the first present invention, it is possible to make divideddisplay of small screens easily.

According to the second present invention, it is possible to makedivided display of small screens easily even when a use condition ischanged.

1. An image processing apparatus comprising: detecting means fordetecting coordinates on a small screen for displaying a small imageincluding predetermined coordinates on a large screen for displaying alarge image comprising a plurality of small images disposed atpredetermined positions, said coordinates on said small screencorresponding to said predetermined coordinates on said large screen;reading means for reading a pixel value of a pixel of the predeterminedsaid small image, said pixel being located at a position correspondingto the coordinates on said small screen detected by said detectingmeans; and outputting means for outputting said pixel value read by saidreading means as a pixel value of a pixel of said large image, saidpixel being located at a position corresponding to said predeterminedcoordinates on said large screen.
 2. The image processing apparatus asclaimed in claim 1, further comprising storing means for storing a tablein which the coordinates on said large screen, the coordinates on saidsmall screen including the coordinates on said large screen, saidcoordinates on said small screen corresponding to the coordinates onsaid large screen, and information for identifying said small image tobe displayed on said small screen are associated with each other,wherein said detecting means detects the coordinates on said smallscreen including said predetermined coordinates on said large screen,said coordinates on said small screen corresponding to saidpredetermined coordinates, from said table; and said reading means readsthe pixel value of the pixel of said small image identified by theinformation for identifying said small image, said information beingassociated with said predetermined coordinates in said table, said pixelbeing located at the position corresponding to the coordinates on saidsmall screen detected by said detecting means.
 3. The image processingapparatus as claimed in claim 1, wherein said small image is an imagecorresponding to a picked-up image obtained as a result of image pickupby an image pickup device.
 4. An image processing method comprising: adetecting step for detecting coordinates on a small screen fordisplaying a small image including predetermined coordinates on a largescreen for displaying a large image comprising a plurality of smallimages disposed at predetermined positions, said coordinates on saidsmall screen corresponding to said predetermined coordinates on saidlarge screen; a reading step for reading a pixel value of a pixel of thepredetermined said small image, said pixel being located at a positioncorresponding to the coordinates on said small screen detected by aprocess of said detecting step; and an outputting step for outputtingsaid pixel value read by a process of said reading step as a pixel valueof a pixel of said large image, said pixel being located at a positioncorresponding to said predetermined coordinates on said large screen. 5.A recording medium on which a computer readable program is recorded,said program comprising: a detecting control step for controllingdetecting coordinates on a small screen for displaying a small imageincluding predetermined coordinates on a large screen for displaying alarge image comprising a plurality of small images disposed atpredetermined positions, said coordinates on said small screencorresponding to said predetermined coordinates on said large screen; areading control step for controlling reading a pixel value of a pixel ofthe predetermined said small image, said pixel being located at aposition corresponding to the coordinates on said small screen detectedby a process of said detecting control step; and an outputting controlstep for controlling outputting said pixel value read by a process ofsaid reading control step as a pixel value of a pixel of said largeimage, said pixel being located at a position corresponding to saidpredetermined coordinates on said large screen.
 6. A program that makesa computer perform a process comprising: a detecting control step forcontrolling detecting coordinates on a small screen for displaying asmall image including predetermined coordinates on a large screen fordisplaying a large image comprising a plurality of small images disposedat predetermined positions, said coordinates on said small screencorresponding to said predetermined coordinates on said large screen; areading control step for controlling reading a pixel value of a pixel ofthe predetermined said small image, said pixel being located at aposition corresponding to the coordinates on said small screen detectedby a process of said detecting control step; and outputting control stepfor controlling outputting said pixel value read by a process of saidreading control step as a pixel value of a pixel of said large image,said pixel being located at a position corresponding to saidpredetermined coordinates on said large screen.
 7. An informationprocessing apparatus for generating a first table supplied to an imageprocessing apparatus, said image processing apparatus detecting, from afirst table, coordinates on a small screen for displaying a small imageincluding predetermined coordinates on a large screen for displaying alarge image comprising a plurality of small images disposed atpredetermined positions, said coordinates on said small screencorresponding to said predetermined coordinates on said large screen,reading a pixel value of a pixel of said small image identified byinformation for identifying said small image, said information beingassociated with said predetermined coordinates in said first table, saidpixel being located at a position corresponding to the detectedcoordinates on said small screen, and outputting the read said pixelvalue as a pixel value of a pixel of said large image, said pixel beinglocated at a position corresponding to said predetermined coordinates onsaid large screen, said information processing apparatus comprising:first detecting means for detecting the coordinates on said small screenincluding the coordinates on said large screen, said coordinates on thesmall screen corresponding to the coordinates on said large screen;second detecting means for detecting the information for identifyingsaid small image, said information being associated with the coordinateson said large screen; and generating means for generating said firsttable by storing the coordinates on said large screen, the coordinateson said small screen detected by said first detecting means, and theinformation for identifying said small image detected by said seconddetecting means in association with each other.
 8. The informationprocessing apparatus as claimed in claim 7, wherein: said small image isan image corresponding to a picked-up image obtained as a result ofimage pickup by an image pickup device; and the coordinates on saidlarge screen are associated with information for identifying said imagepickup device having a corresponding image pickup area as theinformation for identifying said small image in each area correspondingto the image pickup area of said image pickup device.
 9. The informationprocessing apparatus as claimed in claim 8, further comprising storingmeans for storing a second table in which coordinates on said smallscreen of said picked-up image for said small screen which image is cutout from said picked-up image after being subjected to correction on abasis of a condition of an optical system of said image pickup deviceand coordinates on said large screen when said picked-up image for saidsmall screen is positioned on the predetermined said small screen, saidcoordinates on said large screen corresponding to the coordinates onsaid small screen, are associated with each other, wherein said firstdetecting means detects coordinates on said small screen correspondingto coordinates on said large screen from said second table, and detectscoordinates on said small screen of said picked-up image before thecorrection from the detected coordinates on said small screen.
 10. Aninformation processing method of an information processing apparatus forgenerating a first table supplied to an image processing apparatus, saidimage processing apparatus detecting, from a first table, coordinates ona small screen for displaying a small image including predeterminedcoordinates on a large screen for displaying a large image comprising aplurality of small images disposed at predetermined positions, saidcoordinates on said small screen corresponding to said predeterminedcoordinates on said large screen, reading a pixel value of a pixel ofsaid small image identified by information for identifying said smallimage, said information being associated with said predeterminedcoordinates in said first table, said pixel being located at a positioncorresponding to the detected coordinates on said small screen, andoutputting the read said pixel value as a pixel value of a pixel of saidlarge image, said pixel being located at a position corresponding tosaid predetermined coordinates on said large screen, said informationprocessing method comprising: a first detecting step for detecting thecoordinates on said small screen including the coordinates on said largescreen, said coordinates on the small screen corresponding to thecoordinates on said large screen; a second detecting step for detectingthe information for identifying said small image, said information beingassociated with the coordinates on said large screen; and a generatingstep for generating said first table by storing the coordinates on saidlarge screen, the coordinates on said small screen detected by a processof said first detecting step, and the information for identifying saidsmall image detected by a process of said second detecting step inassociation with each other.
 11. A recording medium on which a computerreadable program of an information processing apparatus for generating afirst table supplied to an image processing apparatus is recorded, saidimage processing apparatus detecting, from a first table, coordinates ona small screen for displaying a small image including predeterminedcoordinates on a large screen for displaying a large image comprising aplurality of small images disposed at predetermined positions, saidcoordinates on said small screen corresponding to said predeterminedcoordinates on said large screen, reading a pixel value of a pixel ofsaid small image identified by information for identifying said smallimage, said information being associated with said predeterminedcoordinates in said first table, said pixel being located at a positioncorresponding to the detected coordinates on said small screen, andoutputting the read said pixel value as a pixel value of a pixel of saidlarge image, said pixel being located at a position corresponding tosaid predetermined coordinates on said large screen, said programcomprising: a first detecting control step for controlling detecting thecoordinates on said small screen including the coordinates on said largescreen, said coordinates on said small screen corresponding to thecoordinates on said large screen; a second detecting control step forcontrolling detecting the information for identifying said small image,said information being associated with the coordinates on said largescreen; and a generating control step for controlling generating saidfirst table by storing the coordinates on said large screen, thecoordinates on said small screen detected by a process of said firstdetecting control step, and the information for identifying said smallimage detected by a process of said second detecting control step inassociation with each other.
 12. A program of an information processingapparatus for generating a first table supplied to an image processingapparatus, said image processing apparatus detecting, from a firsttable, coordinates on a small screen for displaying a small imageincluding predetermined coordinates on a large screen for displaying alarge image comprising a plurality of small images disposed atpredetermined positions, said coordinates on said small screencorresponding to said predetermined coordinates on said large screen,reading a pixel value of a pixel of said small image identified byinformation for identifying said small image, said information beingassociated with said predetermined coordinates in said first table, saidpixel being located at a position corresponding to the detectedcoordinates on said small screen, and outputting the read said pixelvalue as a pixel value of a pixel of said large image, said pixel beinglocated at a position corresponding to said predetermined coordinates onsaid large screen, said program characterized by making a computerperform a process comprising: a first detecting control step forcontrolling detecting the coordinates on said small screen including thecoordinates on said large screen, said coordinates on said small screencorresponding to the coordinates on said large screen; a seconddetecting control step for controlling detecting the information foridentifying said small image, said information being associated with thecoordinates on said large screen; and a generating control step forcontrolling generating said first table by storing the coordinates onsaid large screen, the coordinates on said small screen detected by aprocess of said first detecting control step, and the information foridentifying said small image detected by a process of said seconddetecting control step in association with each other.